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Abstract 

Markov process is widely applied in almost all aspects of literature, especially important for un- 
derstanding non-equilibrium processes. We introduce a decomposition to general Markov process 
in this paper. This decomposition decomposes the process into 3 independent parts: stationary 
distribution, symmetric detailed-balance part and anti-symmetric breaking detailed-balance part. 
This complete decomposition captures the steady state as well as the dynamics of the process, pro- 
viding an elegant perspective for construction or analyzing problems. In light of the decomposition, 
a unique definition of relative entropy is found to formally separate the effect of detailed-balance 
part and breaking detailed-balance part. We find that the relative Gini entropy production intro- 
duced in the paper is not affected by the non-detailed balance part of the process. This property 
do not holds for other entropy definition in general discrete case. 
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I. INTRODUCTION 



Markov process is widely applied in almost all aspects of sciences, especially in biology, 
from complex networks [2, 3, 19, 26] to evolution dynamics [14, 27, 29], and many others 
[13, 16, 23, 30]. 

In general, we can group Markov process into two categories, detailed balance Markov 
process and non-detailed balance Markov process. The detailed balance Markov process is 
well studied and applied in literature. On the other hand, most systems found in nature 
are not detailed balance ones. Such kind of systems include biological systems, chemical 
systems, economical systems etc. Due to the lack of detailed balance condition, the studies 
for these systems are much harder than detailed balance ones. 

For general Markov process, it is a good way to decompose it into separate parts for 
categorizing and analyzing purpose. Many attempts have been taken to decompose Markov 
process to reveal its underlying structure [21] [18] [7] [31]. In this paper, we introduce a flow 
decomposition view of general Markov process, whose special case in continuous space em- 
braces a recent developed framework in terms of Stochastic differential equation [1, 4, 17, 28]. 

Our decomposition decomposes the process into 3 independent parts: stationary dis- 
tribution, symmetric detailed-balance part and anti-symmetric breaking detailed balance 
part (circulation flow). This complete decomposition captures the steady state as well as 
the dynamics of the process, providing an elegant perspective for construction or analyz- 
ing problems. In this paper, we concentrate on discussing the properties of a class of state 
functions under the decomposition. 

State functions are widely used when considering macroscopic properties of complex sys- 
tems. It is well known that Shannon entropy does not monotonically increase in general 
Markov process [25]. Instead, relative entropy or free energy preserve this monotone prop- 
erty. In addition, [1] demonstrates that in continuous space Markov process, the antisym- 
metric part does not affect the derivative of all definitions of relative entropy. In discrete 
space, however, this claim is no longer valid for all definitions of relative entropy. Among 
them, we find a definition of relative entropy (we name it relative Gini entropy) to formally 
separate the effect of detailed-balance part and breaking detailed balance part in general 
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Markov process, in both discrete and continuous spaces. The specialty of relative Gini en- 
tropy lies in that it is the only definition of relative entropy in discrete space that the effect 
of breaking detailed balance part would vanish. Definitions such as relative Shannon entropy 
do not enjoy this property and will lead to subtle difficulties [12] when analyzing. To have 
a taste of the power of relative Gini entropy, we demonstrate that the relative Gini entropy 
production bound can leads to famous eigen value bound [8]. 

This article is structured as follows. In the second section, we will first review the concept 
of Markov process we discuss in this article. Then we will talk about our flow decomposition 
of Markov process in the third section. The fourth section serves as an example to use the 
framework to analyze the entropy evolution in non-detailed balance process. At last, we will 
talk about connections of our work to others and give conclusion and discussions. 

II. FLOW DECOMPOSITION ON MASTER EQUATION 

In this section, we introduction our flow decomposition method on master equation. We 
will limit our discussion to continuous time Markov process and restrict our process to be 
with a unique stationary distribution which is non-zero over all the points in the state pace. 

A. Master equation 

Assume the discrete state space S = {1, 2, ■ ■ ■ n}. The discrete space Markov process is 
usually described as following master equation [21]. 



Where Qij > 0(i ^ j) are transition rate constants. The master equation have natural inter- 
pretation as follows: the change of probability in a state is equal to the inflow of probability 
from other states minus the outflow from current state to others. Here dtPi(t) corresponds 
to the change of probability in state i. QijPj(t) the inflow from j to i, and — QuPiit) is the 
total outflow from state i. Equation 1 is usually described in matrix form 





dtvit) = Qp(t) 



(2) 
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The stationary distribution can be solved by setting 

= Qtt (3) 

Because Q is not a full rank matrix, solution of Equation 3 must exist and is eigen-vector of 
Q. The solution is unique when Q only have one zero eigen-value. There is an elegant graph 
algorithm to calculate the stationary distribution [21] [31]. Other more intuitive conditions 
on Q also exists to guarantee the uniqueness of the solution. As stated before, we will assume 
such conditions exist for our problem. For futher details of the conditions, readers can refer 
to [11] [25]. 

B. Flow decomposition on master equation 

The flow decomposition of master equation 2 is formally given by the following equations. 

d t p(t) = Qp(t) 

= F FT 1 p(t) 

= [s + A]n- 1 P (t), (4) 

where II = diag[7Ti, n 2 , . . . , vr n ], 

S = (F + F T ) /2, A = (F - F T ) /2 

Here FI is a diagonal matrix with the stationary distribution ir as its elements. Fy = Qy7rj 
is the probability flow from state j to state i in the stationary distribution, thus has zero 
column sum and row sum. S is a symmetric matrix and A is an anti-symmetric matrix, 
both of which have zero column sum and row sum. 

Each part of the decomposition have their own meanings. With the probability conserva- 
tion nature of master equation, we just decompose the probability flow F into two different 
parts. The symmetric detailed balance part S has an equal probability flow between arbi- 
trary pair of states. While the anti-symmetric breaking detailed balance part A stands for 
circulation flux. 

In cases when the circulation flux is vanishing, i.e., A = 0, we can find the process satisfies 
the detailed balance condition. That is why we call S detailed balance part and A breaking 
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FIG. 1. Flow decomposition, a. 2 states decomposition, b. 3 states decomposition. 

detailed balance part. Noting that the degree of freedom of A is zero when n — 2, which 
means that a system of 2 states always meets detailed balance condition. It is only when 
n > 3 that the circulation flux emerges. 

With simple calculation, it can be verified that the total degree of freedom of the three 
parts is same as original Q matrix. In fact, the symmetric matrix S with n(n — l)/2 degree 
of freedom, the anti-symmetric matrix A with (n — l)(n — 2)/2 degree of freedom, and the 
stationary distribution 7r with n — 1 degree of freedom which are mutually independent add 
up to the degree of freedom of Q matrix n 2 — n. 

Figure 1 shows flow decomposition with 2-state system and 3-state system. If there are 
only 2 states, the total degree of freedom of Q-matrix is 2, which include 1 for stationary 
distribution, and 1 for the symmetric flow, noting that in this case anti-symmetric flux 
vanishes. In the 3-state case, the anti-symmetric flux emerges with 1 degree of freedom. The 
detailed balance flow has 3 degree of freedom and stationary distribution has 2. They add 
up to 6 degrees of freedom of the original Q-matrix. 

A straightforward advantage of the decomposition view is that it reveals more detailed 
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information by each part. For example, we can easily construct a series of Markov process 
with same stationary distribution tt by changing S and A. It also gives some connection 
between the process and its dual process. We can get the dual process by reversing anti- 
symmetric flow A. 



III. ABSTRACT FLOW DECOMPOSITION 



Now we introduce an abstract flow decomposition framework here, which includes both 
discrete and continuous space Markov processes as special cases. 
A general Markov process can be formalized as following equation 

d tP (t) = C[^} (5) 

Where C- is called generator, p is the probability density function in continuous space case 
and probability function in discrete space case. Abstract flow decomposition says that we 
can decompose operator C into two operators. 

C = S + A (6) 

S is an self-adjoint operator and A is anti-symmetric operator. They satisfies the following 
properties: 

{Su, v) = (u, Sv) (7) 



(Au,v) = -(u,Av) (8) 

The inner product is Euclid inner product in the space of p. The flow decomposition in 
the discrete case is a special case of this abstract definition. Where Qtt corresponds to the 
operator C, and A and S corresponds to A and S. 

This abstract definition serves as generalization of the finite discrete space decomposition. 
While we can still view S, A and n as detailed balance flow, non-detailed balance flow and 
stationary distribution. We show that this abstract decomposition includes the previous 
framework of decomposition in continuous case in appendix. 
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IV. DYNAMICS IN TERMS OF ENTROPY EVOLUTION 



Entropy is one of the most important and mysterious state function in thermodynamics. 
Discussions on Entropy has been going for hundreds of years and are still going on. How to 
define entropy in non-equilibrium systems is still one of the biggest problems. 

It is well known that Shannon entropy in Markov process is not monotonically increasing. 
While a general definition of relative entropy makes second law of thermodynamics a built- 
in property of master equation described system. Previous work [1] has shown that in 
continuous space case, the derivative of any definition of relative entropy is not affected by 
the breaking detailed part. Nevertheless, this is no longer true in the more general discrete 
space case, where only one special definition of relative entropy (relative Gini entropy) enjoys 
this property. We will discuss relative Gini entropy and show its power by using it to prove 
the famous eigen value bound. 

A. Entropy and relative entropy 

The most famous definition of entropy is the Shannon entropy introduced from informa- 
tion theory and statistical physics [6]. 



However, this definition of entropy is commonly not monotone in general Markov process. 
Accordingly, applying maximum entropy principle with this definition of entropy to general 
process probably will end up with inconsistent conclusions. On the other hand, if we take 
the stationary distribution into account and introduce the definition of relative entropy, the 
monotone property will be preserved and the second law of thermodynamics (arrow of time) 
will turn out to be a built-in property for master equation described processes. 

For a discrete state space with n state, the relative / entropy is defined as follows 



Here / is a convex function. Different choices of / give different definitions of entropy. The 
most common one is relative Shannon entropy. 



n 




(9) 




(10) 
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FIG. 2. Entropy evolution, a. Monotonic evolution of Relative Shannon Entropy(Solid) and 
Non-monotonic evolution of Shannon Entropy (Dashed). Different colors are for different initial dis- 
tributions, b. System evolution in Shannon entropy contours, c. System evolution in Relative 
Shannon entropy contours. 



The monotone property of relative entropy is well known in literature, the following 
Figure. 2 serves as an example to illustrate the underlying reason of how this happens, 
(a) Compares the evolution of the non- monotone Shannon entropy (Dashed lines) and the 
monotone Relative Shannon entropy (solid lined). Different colors are for different initial 
distribution, (b)and(c) further illustrates how (a) happens: The non-uniform stationary dis- 
tribution cause a shifting of the fix state from the center, leading to the sometimes increasing 
and sometimes decreasing Shannon entropy. 

8 



B. Relative Gini Entropy 



Now we introduce a special definition of relative entropy, the relative Gini entropy. This 
definition formally separates the effects of detailed balance part and breaking detailed bal- 
ance part in current decomposition view, suggesting their dissipative and conserved nature 
separately. It is worth to mention that Gini entropy is the unique choice to have this prop- 
erty in both discrete and continuous space cases. In light of relative Gini entropy, we rewrite 
the proof of famous eigen value bound to have a taste of the importance of this definition. 

Relative Gini entropy [20], which is a special case of Tsallis entropy [5], chooses f(x) = x 2 
and is defined as follows 

8=1 \ */ 

Using our definition of flow decomposition, we can put the entropy production of relative 
Gini entropy as follows 

T 



$£«(p) = -2(!) (S + A)( 



V 



TT/ V 7T / ( 12 j 



Here - = [— , — , • • • , 2^1 T . Remember that S is symmetric negative semi-definite matrix 

7T L 7T1 7 7T2 7 7 7l n J 

and A is anti-symmetric matrix and they both have zero column sum and role sum. We 
have 

^(p) = -2(!) r s(!)>0 (13) 

Equation 13 says that relative Gini entropy will monotonically increase over time, this is 
exactly the statement of second law of thermodynamics. In addition, the derivative of 
relative Gini entropy production is not affected by the breaking detailed balance part of the 
process. This property indicates the conservative nature of the dynamics associated with the 
breaking detailed balance part. It is worth mentioning that this specialty of relative Gini 
entropy only exists in discrete case. When it comes to continuous case, other definition of 
relative entropy can also eliminate the effects of breaking detailed balance part [1]. 

In the following, we give out the proof of eigen value bound in terms of decomposition 
and relative Gini entropy view. 
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Theorem IV. 1. The evolution of relative Gini entropy can be bounded by 



S°(p,t)>S°(p,0)e 



-\it 



(14) 



Where A 2 is the second smallest absolute eigen value of the following matrix G 



G = diag(v^)" 1 S diag(v^) 



-i 



(15) 



Proof. We know that evolution of the entropy satisfies the following equation. 

S"(p,t) = S<'( P ,0)+ f ' d t S°(p)dt 
Jo 



(16) 



If we can bound the entropy production dtS g (p), we can bound the rate of convergence of 
Markov process. 
Since we have 



~(H T »( 



p 



p 

7T 

T 



\fn G I . - 

■> J \ V " 



P 



(17) 
(18) 

(19) 



We can find the length (norm) of the vector — V^j is related to current relative Gini 
entropy 



, x 2 N 2 N N 

v v 7 i=l 1 i=l i=l 



(20) 



For the new matrix G, we can find it is also negative semi-definite, and have a eigen-value 
of 0. 



gV^ = o 



Assume the eigen decomposition of G as follows 



(21) 



G = -U diag([Ai, A 2 , • • • , Ajv])U j , A^ > • • • > A 2 > Ai = 0, Ui = (22) 
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Then the entropy production becomes 

T 



d t S 9 (p) 



P 



7T 



yfr) Udiag([Ai,A 2 ,-- - ,Atv])U 3 



TIP 



7T 



7/ T diag([Ai,A 2 ,--- An])v 



N 



(23) 
(24) 
(25) 



i=l 



Where yi is the projection of — over Uj, we can find y have following properties 

= S 9 {p) (26) 



P 



Vi = Ui 



p 



71 



p 







(27) 



"K J VV 71 " 

Combining above results, we can state the eigen-value bound of entropy production as 
Equation. 14. □ 

Due to the nice property of relative Gini entropy, we derive the well known eigen-value 
bound of convergence without any struggle. We find that the previous proofs [10] [8] of this 
result implicitly applied the idea of decomposition. This is usually given by the concept of 
dual process. Previous study on non-detailed balance process mostly reduce the problem 
to the detailed balance part of the process. Example of such works include eigen-value 
bound[10] and log-Soblev bound[8] of Markov process. Because anti-symmetric part is not 
explicitly used, the decomposition can be used implicitly by concept of process and its dual 
process. 



V. DISCUSSION 



The abstract flow decomposition we introduced here can be viewed as a generalization 
of previous work done on continuous space case [1] [4]. People once doubted whether similar 
method can be applied to discrete space case as well. We at this point illustrated that 
discrete space case and continuous space case can be unified with abstract flow decomposition 
view. Since continuous case can be seen as a limit of general discrete case, the conclusions 
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we obtain in discrete case should always be valid in continuous case, but not vice versa. 
Relative Gini entropy serves as an example here, which is the unique choice to eliminate the 
effects of breaking detailed balance part in discrete case. While in continuous case, all kinds 
of relative / entropy can do. Another important thing is that in discrete case the stationary 
distribution is not ambiguous, while in continuous case, the emergence of multiplicative noise 
will lead to ambiguous stationary distributions [22]. 

The decomposition view of Markov process can act as guide lines for designing algorithms 
or analyzing problems. Though finding rules in general process is usually difficult, there 
are already some special cases that can be understood clearly under this view [24] [9] . More 
rules are waiting to be discovered along the line. It is worthy to mention that the breaking 
detailed balance part can be further decomposed into smaller circulation parts [7, 15], which 
may be useful for more detailed analysis. 

There has been similar work done on decomposition of Markov process in literature [31]. 
However, they focus on the steady states. Since our decomposition here is complete, the sta- 
tionary distribution, the symmetric detailed balance part, and the anti-symmetric breaking 
detailed balance part together make the whole process, including the information of both 
steady state and dynamics. 

The entropy evolution is also the main concern in literature [12] [7] . For processes with non- 
uniform distribution as stationary distribution, one major understanding is to decompose 
the change of the non-monotone Shannon entropy into two parts: non-negative entropy 
production within the system and transfer of heat across the boundary. This "outsider" view 
is in contrast with our "insider" view, which directly consider the relative entropy within the 
system. Such direct treatment should be easier in dealing with some complicated problems. 
Furthermore, relative Gini entropy play a quite special role in light of current decomposition 
view, which separate the effects of detailed balance part and breaking detailed balance part, 
leading to simplification for analysis. 
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VI. CONCLUSION 

In this work, we introduce a flow decomposition view in dealing with Markov process. It 
combines the existing decomposition in continuous Markov process and the current decom- 
position in discrete case. This decomposition view captures both steady states and dynamics 
structure of Markov process. In light of the decomposition, we find a special definition of 
relative entropy which can formally eliminate the effects of breaking detailed balance part 
and lead to dramatic simplification of analysis. We suggest this view as a good perspective 
for dealing with Markov process, further study should be carried on along the line. 
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Appendix A: Flow decomposition in continuous process 

The decomposition on continuous space case is put forward by Ao [17] with the follow- 
ing special form of Fokker-Planck equation, which is the continuous counterpart of master 
equation. 



Where x is a n dimensional variable, D(x) is the symmetric positive semi-definite diffusion 
matrix, G(x) is anti-symmetric matrix and <p(x) is the potential function. It is easy to see 
the stationary distribution is the famous Boltzmann-Gibbs distribution 




(Al) 



irlx) oc e 



(A2) 



The abstract flow decomposition of Equation Al is given as follows 



S(u) = V T {D [V(f> + V] nu} 



(A3) 
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A(u) = V T {G [V0 + V] mi} (A4) 

We now prove that operator S and A do satisfies the property of symmetry and anti- 
symmetry in our definition. 

Proof. The general idea of proof 



aV(x) = -vr(x)<9>(x) (A5) 

• Maybe we need to swap entropy % and j 

• Use integral by parts when necessary 

Here d l mean taking partial derivative to i — th dimension. 

(Sf,9) (A6) 



^A,/(a;)7r(x)9V(a:) + ^A J /(x)^(/(x)7r(x)) dx (A7) 

i L j j 

= Yl 1 [D l3 f(x)ir(x)d^(x) + Dijd j (f(x)rr(x))] dx (A8) 

= j 9{x)& [-D i:j f{x)d j ir(x) + Dijd* (/(x)vr(x))] dx (A9) 

=(f,Sg) by symmetry (A13) 

For ^4, the proof is exactly the same except using the fact that G is antisymmetric matrix 

in swap index step. □ 
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By far we have shown that our definition of flow decomposition connects to the previous 
works of decomposition in continuous diffusion process. This also gives another justification 
of the flow decomposition view since we can have a framework working with both discrete 
and continuous state space. And our definition of decomposition in finite discrete Markov 
process can be viewed as a discrete space counterpart of the continuous decomposition by 
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